PatchCensor: Patch Robustness Certification for Transformers via Exhaustive Testing

In the past few years, Transformer has been widely adopted in many domains and applications because of its impressive performance. Vision (ViT), a successful well-known variant, attracts considerable attention from both industry academia thanks to record-breaking performance various vision tasks. However, ViT is also highly nonlinear like other classical neural networks could be easily fooled by natural adversarial perturbations. This limitation pose threat deployment real industrial environment, especially safety-critical scenarios. How improve robustness thus an urgent issue that needs addressed. Among all kinds robustness, patch defined as giving reliable output when random input domain perturbed. The perturbation corruption, such part camera lens being blurred. It distribution shift, object does not exist training data suddenly appearing camera. And worst case, there malicious attack aims fool prediction machine learning model arbitrarily modifying pixels within restricted region image. kind called physical it believed more than digital attack. Although some work on improvement Convolutional Neural Network (CNN), related studies counterpart are still at early stage usually much complex with far parameters. harder assess mention provide provable guarantee. this work, we propose PatchCensor, aiming certify applying exhaustive testing. We try guarantee considering Unlike empirical defenses against patches may adaptively breached, certified robust approaches can accuracy arbitrary attacks under certain conditions. existing certifications mostly based training, which often requires substantial efforts sacrifice normal samples. To bridge gap, PatchCensor seeks whole system detecting abnormal inputs instead asking give results for every input, inevitably compromise accuracy. Specifically, each tested voting over multiple inferences different mutated masks, where least one inference guaranteed exclude patch. seen complete-coverage testing, statistical test time. Our comprehensive evaluation demonstrates able achieve high ( e.g. 67.1% ImageNet 2%-pixel patches), significantly outperforming state-of-the-art techniques while achieving similar clean (81.8% ImageNet). same vanilla models. Meanwhile, our technique supports flexible configurations handle sizes simply changing masking strategy.